ABSTRACT
The coronavirus pandemic has highlighted the importance of developing intelligent robotics to prevent infectious disease spread. Human-machine interfaces (HMIs) give a chance of interactions between users and robotics, which play a significant role in teleoperating robotics. Conventional HMIs are based on bulky, rigid, and expensive machines, which mainly focus on robots/machines control, but lack of adequate feedbacks to users, which limit their applications in conducting complicated tasks. Therefore, developing closed-loop HMIs with both accurate sensing and feedback functions is extremely important. Here, we present a closed-loop HMI system based on skin-integrated electronics, whose electronics compliantly interface with the whole body for wireless motion capturing and haptic feedback via Bluetooth, Wireless Fidelity (Wi-Fi), and Internet. The integration of visual and haptic VR via skin-integrated electronics together into a closed-loop HMI for robotic VR demonstrates great potentials in noncontact collection of bio samples, nursing infectious disease patients and many others.
ABSTRACT
Wearing masks has been a recommended protective measure due to the risks of coronavirus disease 2019 (COVID-19) even in its coming endemic phase. Therefore, deploying a "smart mask" to monitor human physiological signals is highly beneficial for personal and public health. This work presents a smart mask integrating an ultrathin nanocomposite sponge structure-based soundwave sensor (≈400 µm), which allows the high sensitivity in a wide-bandwidth dynamic pressure range, i.e., capable of detecting various respiratory sounds of breathing, speaking, and coughing. Thirty-one subjects test the smart mask in recording their respiratory activities. Machine/deep learning methods, i.e., support vector machine and convolutional neural networks, are used to recognize these activities, which show average macro-recalls of ≈95% in both individual and generalized models. With rich high-frequency (≈4000 Hz) information recorded, the two-/tri-phase coughs can be mapped while speaking words can be identified, demonstrating that the smart mask can be applicable as a daily wearable Internet of Things (IoT) device for respiratory disease identification, voice interaction tool, etc. in the future. This work bridges the technological gap between ultra-lightweight but high-frequency response sensor material fabrication, signal transduction and processing, and machining/deep learning to demonstrate a wearable device for potential applications in continual health monitoring in daily life.